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[Name of Document] SPECIFICATION 
[Title of the Invention] 

IMAGE PROCESSING APPARATUS AND METHOD , AND DISTRIBUTION 
MEDIUM 
[Claims ] 

[Claim 1] An image processing apparatus for generating 
a motion signal for controlling motion in accordance with an 
image, said image processing apparatus comprising: 

detection means for detecting a motion vector from said 
image signal; and 

generation means for generating said motion signal from 
a motion vector detected by said detection means. 

[Claim 2] An image processing apparatus according to 
claim 1, wherein said detection means detects said motion 
vector for each block composed of a plurality of pixels 
positioned at predetermined positions within a frame. 

[Claim 3] An image processing apparatus according to 
claim 1, wherein said generation means generates a 
horizontal component, a vertical component, a magnification 
component, and a rotation component from the motion vector 
detected by said detection means, and generates said motion 
signal from these four components. 

[Claim 4] An image processing method for use with an 
image processing apparatus for generating a motion signal 
for controlling motion in accordance with an image, said 
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image processing method comprising: 

a detecting step for detecting a motion vector from 
said image signal; and 

a generating step for generating said motion signal 
from a motion vector detected in said detecting step. 

[Claim 5] A distribution medium for providing a 
computer-readable program to an image processing apparatus 
for supplying a motion signal for controlling motion in 
accordance with an image in order to execute: 

a detecting step for detecting a motion vector from 
said image signal; and 

a generating step for generating said motion signal 
from a motion vector detected in said detecting step. 
[Detailed Description of the Invention] 

[0001] 

[Industrial Field of the Invention] 

The present invention relates to an image processing 
apparatus and method , and a distribution medium. More 
particularly, the present invention relates to an image 
processing apparatus and method for detecting a motion 
vector from an input image signal and for generating a 
motion signal for controlling the motion of a chair for an 
observer who views an image from the detected motion vector, 
and to a distribution medium therefor. 

[0002] 



- 3 - 

[Description of the Related Art] 

An apparatus in which, when an observer is viewing an 
image, verisimilitude is improved by moving, in accordance 
with the image, the motion of a chair on which the observer 
is sitting, is known. A motion signal, which is supplied to 
such an apparatus to cause a chair to move, is generated 
from data obtained by a sensor for detecting an angle at the 
same time when an image is captured, or a motion signal is 
generated through a manual operation by observing the 
captured image by a person and by predicting motion by that 
person. 

[0003] 

[Problems to be Solved by the Invention] 
In the above-described apparatus, when a motion signal 
for a chair is generated by a sensor, data for generating 
the motion signal must be recorded at the same time the 
image is captured. Therefore, it is not possible to 
generate a motion signal by using an image which has already 
been captured. 
[0004] 

Also, when a person observes a captured image and a 
motion signal is generated, a manual operation is required, 
presenting a problem in that processing takes a long time. 

[0005] 

The present invention has been achieved in view of such 



# 
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circumstances. The present invention aims to easily 
generate a motion signal also from an image which has 
already been captured by detecting a motion vector from an 
input image and by generating a motion signal from the 
detected motion vector. 
[0006] 

[Means for Solving the Problems] 

The image processing apparatus as set forth in claim 1 
comprises detection means for detecting a motion vector from 
an image signal; and generation means for generating a 
motion signal from the motion vector detected by the 
detection means. 



The image processing method as set forth in claim 4 
comprises a detecting step for detecting a motion vector 
from an image signal; and a generating step for generating a 
motion signal from the motion vector detected by the 
detection means. 



The distribution medium as set forth in claim 5 
provides a computer-readable program to an image processing 
apparatus in order to execute a process comprising: a 
detecting step for detecting a motion vector from an image 
signal; and a generating step for generating a motion signal 
from the motion vector detected in the detecting step. 



[0007] 



[0008] 
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[0009] 

In the image processing apparatus as set forth in claim 
1, the image processing method as set forth in claim 4, and 
the distribution medium as set forth in claim 6, a motion 
vector is detected from an image signal , and a motion signal 
is generated from the detected motion vector. 

[0010] 

[Description of the Embodiment] 

Fig. 1 is a block diagram showing the construction of 
an embodiment of an image processing apparatus of the 
present invention. An image signal supplied from an image 
tape recorder (not shown), etc., is supplied to a display 
section 1 formed of a display and an image processing 
apparatus 2. The image processing apparatus 2 generates a 
control signal (motion signal) for driving a driving section 
3. 

[0011] 

The image processing apparatus 2 comprises a feature 
quantity detection section 11 and a feature quantity 
processing section 12. A signal input to the image 
processing apparatus 2 is input to the feature quantity 
detection section 11 whereby feature quantity (to be 
described later) is detected and is output to the feature 
quantity processing section 12. The feature quantity 
processing section 12 computes a motion signal to be 
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supplied to the driving section 3 from the input feature 
quantity. The driving section 3 drives (moves) a chair on 
which an observer who observes an image sits in accordance 
with the input motion signal. This point will be described 
later. 

[0012] 

Fig. 2 is a block diagram showing the construction of 
the feature quantity detection section 11. The image signal 
input to the feature quantity detection section 11 is 
delayed by an amount corresponding to one frame by a delay 
section 21 , after which it is supplied to a frame buffer 22- 
1 and is also supplied to a frame buffer 22-2. Reading 
sections 23-1 and 23-2 read an image signal from the 
corresponding frame buffers 22-1 and 22-2 in accordance with 
a predetermined pattern stored in a memory 24, and output 
the image signal to a motion vector detection section 25. 

[0013] 

The motion vector detection section 25 detects a motion 
vector from the supplied image signal and outputs the motion 
vector to a feature quantity computation section 26. The 
feature quantity computation section 26 computes feature 
quantity from the input motion vector. 

[0014] 

Next, the operation of the feature quantity detection 
section 11 shown in Fig. 2 is described. At time t, the 



image signal input to the feature quantity detection section 
11 is supplied to the delay section 21 and the frame buffer 
22-2. The frame buffer 22-2 stores input image signals for 
one frame. Since the delay section 21 delays the image 
signal by an amount corresponding to one frame, at time t, 
an image signal at time t-1 earlier than time t (an image 
signal one frame before that at time t) is stored in the 
frame buffer 22-1. The image signal at time t-1, stored in 
the frame 22-1, is read by the reading section 23-1, and the 
image signal at time t, stored in the frame buffer 22-2 , is 
read by the reading section 23-2. 
[0015] 

The reading sections 23-1 and 23-2 read an image signal 
in a portion corresponding to the pattern stored in the 
memory 24 from among the image signals stored in the 
corresponding frame buffers 22-1 and 22-2. Here, a pattern 
stored in the memory 24 is described with reference to Fig. 
3. 

[0016] 

Fig. 3 shows an example of a pattern stored in the 
memory 24. In the case of those portions of the pixels 
forming one frame, which are not related to motion, for 
example, image signals captured by a video camera mounted to 
an automobile, such as that shown in Fig. 4, the hood 
portion of the automobile is determined to be an area which 
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is not related to motion , and the pixel positioned at the 
center of the area from which that area has been removed is 
denoted as a convergence point P. For example , 2 5 
representative points Q (including the convergence point) 
which are symmetrical horizontally and vertically about the 
convergence point P are set. For each representative point 
Q, a block B, composed of a predetermined number of pixels, 
with the representative point Q being positioned at the 
center, for example, a block B composed of 33 x 33 pixels, 
is set. In the memory 24, the coordinates of each 
representative point Q within the image plane of such a 
frame, and the size of the block B are stored as a pattern. 
[0017] 

The reading section 23-1 reads pixel data corresponding 
to the pattern such as that described above stored in the 
memory 24, that is, pixel data within each block B, in 
accordance with the coordinates of the representative point 
Q and the size of the block B from among the image signals 
at time t-1 stored in the frame buffer 22-1, and outputs the 
pixel data, as the data of the reference block, to the 
motion vector detection section 25. In a similar manner, 
the reading section 23-2 reads pixel data corresponding to 
the pattern stored in the memory 24 from among the image 
signals at time t stored in the frame buffer 22-2, and 
outputs the pixel data, as the data of the retrieval block, 
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to the motion vector detection section 25. 
[0018] 

The motion vector detection section 25 detects the 
motion vector at each representative point Q by performing 
block matching by using the input data of the reference 
block and the input data of the retrieval block. Therefore/ 
in the case of this example, 25 motion vectors are detected. 

[0019] 

The feature quantity computation section 26 computes, 
based on the equations shown below, a total of four 
components of a horizontal component u, a vertical component 
v, a magnification component v zoom/ and a rotation component 
v rot of the niotion of the frame (image plane) by using the 25 
motion vectors detected by the motion vector detection 
section 25. 

Horizontal component u = (1/n) 2u ± ... (1) 
Vertical component v = (1/n) 2v ± ... (2) 
Magnification component v zoom = (1/n) 2v 20omi /d ± ... (3) 
Rotation component v rot = (1/n) Iv^/di ... (4) 
where the subscript i indicates the number affixed to the 
representative point Q ± , which varies from 1 to 25 in this 
example, and n indicates the number of representative points, 
which is 25 in this example. The values obtained by 
equations (1) to (4) are the average value of each component 
u, v, v zoom , and v rot obtained by the motion vector detection 
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section 25. 
[0020] 

The relationship among the above-mentioned components u, 
v ' v zoom' and v rot is described with reference to Fig. 5. The 
horizontal component of a motion vector T of a 
representative point Q ± for the object of processing is 
denoted as u ± , and the vertical component is denoted as v ± . 
d ± is a scalar quantity indicating the distance from the 
convergence point P to the representative point Q ± . (Px, 
Py) indicates the coordinates of the convergence point P, 
and the distance up to the representative point Q ± of the 
coordinates (QjX, Q ± y) is computed with these coordinate 
values as a reference. 

[0021] 

The component (u if v ± ) of this motion vector T is a 
component when the representative point Q ± is assumed to be 
the origin. The component in a direction parallel to the 
straight line which connects the convergence point P to the 
representative point Q ± , of the motion vector T is denoted 
as v zoomi , and the component in a direction perpendicular to 
the straight line which connects the convergence point P to 
the representative point Q ± is denoted as v roti . Also, the 
angle formed between the straight line which connects the 
convergence point P to the representative points Q ± and the 
motion vector T is denoted as 0. At this time, v zoomi and 



v roti are determined based on the following equations. 

V Z oomi = ("i 2 + V^^/^COSO ... (5) 

v roti = <"i 2 + v ± 2 ) (1/2) sine ... (6) 
[0022] 

The feature quantity computation section 26 computes 
four component data u, v, v zoom , and v rot , as feature quantity f 
by using equations (1) to (4) from the motion vector output 
from the motion vector detection section 25. The computed 
four component data u, v, v zoom , and v rot are output to the 
feature quantity processing section 12 (Fig. 1). 

[0023] 

Here, for example , when an observer is made to 
virtually experience the feeling of riding in an automobile , 
the types of forces (motions) which should be applied to the 
chair on which the observer is sitting are considered. 
Examples of forces applied to the chair of the automobile 
include a force for representing the forward-rearward 
inclination of the surface of the road when the automobile 
is running on a sloping road, such as, a grade, a force for 
representing vibrations which are received from the surface 
of the road when the automobile is running on a bumpy road, 
and a force for representing the inclination in the left- 
right direction of the surface of the road when the 
automobile is running on a sloping road. 

[0024] 
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These forces are stimuli (actual stimuli) which are 
actually applied to an automobile to which a video camera 
which has captured an image is mounted. Forces similar to 
these actual stimuli may be applied to the chair for the 
observer who observes the image. For example , if the actual 
stimulus is a right-inclined force, a right-inclining force 
may be applied to the chair for the observer as well. These 
forces are forces which are applied without being influenced 
by the physical condition of the chair for the observer, for 
example, the condition in which the chair can be moved back 
and forth only within a predetermined range. 

[0025] 

In contrast, a force for representing the centrifugal 
force when the automobile is turning on a curve, a force for 
representing the inertial force during acceleration and 
deceleration, and a force for representing the yawing of the 
automobile in a curve must be applied as simulated forces 
(simulated stimuli) because the center of gravity of the 
chair on which the observer is sitting cannot be moved due 
to physical conditions. For example, since the chair cannot 
be moved forward in an accelerated manner so as to represent 
the inertial force during acceleration, a simulated stimulus 
must be applied to the chair for the observer. 

[0026] 

The relationship among the force concerning the above- 
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mentioned actual stimulus and the simulated stimulus, the 
component of the motion signal which is actually applied to 
the chair for the observer, and the four components computed 
by the feature quantity computation section 26 is shown 
below. 

Actual stimulus 



Component to Motion signal 

be represented component 



Relationship 
with 4 components 



Forward- rearward 

inclination of 
surface of road 



pitch 



Low- frequency component 
of Zv 



Vibrations received 
from surface of road 



High-frequency component 
of -Zv 



Left-right 
inclination of 
surface of road 



roll 



-Zv 



rot 



Simulated stimulus 



Component to Motion signal 

be represented component 



Relationship 
with 4 components 



Centrifugal force 



roll 



in curve 



Centrifugal force 



pitch 



Low- frequency components 
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due to acceleration 



of dv, 



zoom 



,/dt 



and deceleration 



Automobile 



yaw 



-u 



yawing in 



curve 



[0027] 



The feature quantity processing section 12 generates a 
motion signal to be supplied to the driving section 3 (Fig. 
1) by using the above-described relationships. Fig. 6 is a 
block diagram showing the construction of the feature 
quantity processing section 12. Among the four components 
output from the feature quantity detection section 11 , the 
rotation component v rot is input to an adder 31-1, the 
horizontal component u is input to an adder 31-2 and a code 
inverter 32-1 , the vertical component v is input to an adder 
31-3, and the magnification component v zoom is input to an 
adder 31-5 and a delay unit 33-3. The data output from the 
adder 31-1 is delayed by an amount corresponding to one 
clock by the delay unit 33-1 , after which it is fed back and 
is input to the adder 31-1. In a similar manner, the data 
which is output from the adder 31-3 is delayed by an amount 
corresponding to one clock by a delay unit 33-2 , after which 
it is fed back and is input to the adder 31-3 as well. 



[0028] 



The data output from the delay unit 33-1 is input to 
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th e adder 31-2, and the data output from the delay unit 33-2 
is output to an HPF (High-Pass Filter) 34 via a code 
inverter 3 2-2 and is also output to an adder 31-4 via an LPF 
(Low-Pass Filter) 35-1. The magnification component v zoom 
which is delayed by an amount corresponding to one clock by 
the delay unit 33-3 is subtracted from the magnification 
component v zoom input to an adder 31-5, and the resulting 
component is input to an adder 31-4 via an LPF 35-2. 
[0029] 

Next, a description is given of the computation of the 
motion signal components "roll", "yaw", "z", and "pitch", 
performed by the feature quantity processing section 12. 
The rotation component v rot input to the feature quantity 
processing section 12 is input to the adder 31-1. The adder 
31-1 adds together the rotation component v rot which is input 
at time t and the data which is output one unit time before 
at time t-1 from the delay unit 33-1. The adder 31-1 
computes the motion signal component "roll" (2v rot ) which 
represents the left-right inclination of the surface of the 
road by accumulating (integrating) the rotation component 
v rot in this manner. However, since the motion signal 
component "roll" representing the left-right inclination of 
the surface of the road is -2v rot , the adder 31-2 uses, for 
computation, data such that the code of the data input from 
the delay unit 33-1 is inverted. 



[0030] 

The motion signal component "roll" (horizontal 
component u) is also used to represent the centrifugal force 
in a curve. Accordingly, the adder 31-2 computes the motion 
signal component "roll" to be supplied to the driving 
section 3 by adding together (subtracting the output of the 
delay unit 3 3-1 from the horizontal component u) the data 
such that the code of the data input from the delay unit 33- 
1 is inverted and the horizontal component u. 

[0031] 

Since the motion signal component "yaw" of the yawing 
of the automobile in a curve is obtained by inverting the 
value of the horizontal component u, the feature quantity 
processing section 12 computes the motion signal component 
"yaw" by causing the code inverter 32-1 to invert the code 
of the value of the input horizontal component u. 

[0032] 

The adder 31-3 adds together the vertical component v 
which is input at time t and the vertical component v which 
is output one unit time before at time t-1 from the delay 
unit 33-2. In this manner, the vertical component v is 
accumulated (integrated) by the adder 31-3. Then, the data 
which is accumulated by the adder 31-3 and the delay unit 
33-2 is input to the code inverter 32-2 whereby the code is 
inverted, and further, only the high-frequency components 



are extracted by the HPF 34. In this manner , the motion 
signal component z representing the vibrations received from 
the surface of the road is computed. 
[0033] 

Furthermore, the data output from the delay unit 33-2 
is also output to the LPF 35-1 whereby the low-frequency 
components are extracted. In this manner, the motion signal 
component "pitch" representing the forward-rearward 
inclination of the surface of the road is computed. The 
motion signal component "pitch" is also used as a motion 
signal component representing the inertial force by 
acceleration and deceleration. For this reason, the motion 
signal component "pitch" output from the LPF 35-1 is added 
to the motion signal component "pitch" representing the 
inertial force by the adder 31-4. 

[0034] 

The motion signal component "pitch" representing the 
inertial force is computed from the magnification component 
v zoom input to the feature quantity processing section 12. 
The magnification component v zoom input to the feature 
quantity processing section 12 is input to the adder 31-5 
and the delay unit 33-3. A magnification component v zoom t 
which is input at time t and a magnification component 
v zoom t-l at time t-1, delayed by one unit time by the delay 
unit 33-3, are input to the adder 31-5. The adder 31-5 
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dif ferentiates the magnification component v zoom by 
subtracting the magnification component v zoom t-l at time t-1 
from the magnification component v zoom t input at time t. 
Then, the LPF 35-2 extracts the low-frequency components 
from the value output from the adder 31-5, thereby computing 
the motion signal component "pitch" representing the 
inertial force by acceleration and deceleration. 
[0035] 

The adder 31-4 adds together the value output from the 
LPF 35-1 and the value output from the LPF 3 5-2, thereby 
computing the motion signal component "pitch" to be supplied 
to the driving section 3. 

[0036] 

An example of the driving section 3 is shown in Figs. 7 
and 8. Fig. 7 is a side view of the driving section 3. Fig. 
8 is a view when the driving section 3 is seen from above. 
The driving section 3 comprises six pistons 41-1 to 41-6, 
with a base 42 being supported by these pistons. The base 
42 has a chair 43 fixed thereto so that an observer 44 may 
sit on this chair 43. 

[0037] 

The pistons 41-1 to 41-6 are capable of extending and 
retracting along their central axes. As a result of the 
extending and retracting motion by the pistons 41-1 to 41-6, 
the base 42 jerks, and furthermore, the chair 43 fixed to 



-19- 



the base 42 jerks. A signal for controlling the pistons 41- 
1 to 41-6 is generated and supplied by the feature quantity 
processing section 12 in a manner as described above. 
[0038] 

As described above, in the present invention, the 
motion signal component is computed from the motion vector 
obtained from the image. This makes it possible to obviate 
the need to generate the motion signal component in advance 
in order to input it. In addition, it is possible to easily 
generate a motion signal component from the image in which a 
motion signal component is not generated. 

[0039] 

In this specification, examples of distribution media 
for providing a computer program for executing the above- 
described processing to a user include an information 
recording medium, such as a magnetic disk or a CD-ROM, and 
transmission media by a network, such as a the Internet or a 
digital satellite. 

[0040] 

[Advantages ] 

According to the image processing apparatus as set 
forth in claim 1, the image processing method as set forth 
in claim 4, and the distribution medium as set forth in 
claim 6, a motion vector is detected from an image signal, 
and a motion signal is generated from the detected motion 
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vector. This makes it possible to easily generate a motion 
signal to be supplied to the apparatus for causing a chair 
for an observer to jerk in accordance with the image, 
[Brief Description of the Drawings] 
[Fig. 1] 

Fig. 1 is a block diagram showing the construction of 
an embodiment of an image processing apparatus of the 
present invention . 

[Fig. 2] 

Fig. 2 is a block diagram showing the construction of a 
feature quantity detection section 11 of Fig. 1. 
[Fig. 3] 

Fig. 3 shows a pattern stored in a memory 24 of Fig. 2. 
[Fig. 4] 

Fig. 4 shows an image to be processed. 
[Fig. 5] 

Fig. 5 is an illustration of a vector to be computed. 
[Fig. 6] 

Fig. 6 is a block diagram showing the construction of a 
feature quantity processing section 12 of Fig. 1. 
[Fig. 7] 

Fig. 7 is a side view of a driving section 3. 
[Fig. 8] 

Fig. 8 is a view when the driving section 3 is seen 
from above. 




-21- 

[ Reference Numerals] 

2 image processing apparatus , 11 feature quantity 
detection section, 12 feature quantity processing section, 
21 delay section, 22 frame buffer, 23 reading section, 
24 memory, 25 motion vector detection section, 26 
feature quantity computation section, 31 adder, 32 code 
inverter, 33 delay unit, 34 HPF, 35 LPF 



